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What is Gaming Al and Why is it Important? 


- Simulation of Human Intelligence in Non-Playing Characters (NPC) 
- CPU Bots in games such as: Chess, Go, Starcraft, Hearthstone, League of Legends, etc. 
- Interactive Characters in MMORPG: Runescape, Maplestory, World of Warcraft, Tera, etc. 
- Machine Learning / Deep Learning in Gaming Al 
- AlphaGo considered a milestone 
- Games provide a very good environment for testing and getting feedback 
- Many problems are complicated and can be translated to real word problems 


- In most cases simple to quantify 


Creating A.I. for Games 


1) Identifying the challenge / task you want your A.I. to embody 
a) Ex: An object to test mastery over skill 
i) Embodied by: goombas, chess bots, etc. 
2) Identify how the A.I. should respond to expected input 
a) Ex: Told to win tic-tac-toe with present board 
b) Ex2: Told to move to input coordinate as fast as possible 
3) Create A.I. algorithm that can correctly turn input into expected output 


a) Deep Learning" (term normally encompassing neural networks; weighted tree traversals, Markov Chains for 
scripting) 


b) Pathfinding (A" is the preferred method, to be modified to suite game's needs) 
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Deep Learning in Video Game All. 


AlphaGo (most famous) 


a) “Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search 
programs that simulate thousands of random games of self-play” - snippet from article by creators of AlphaGo: 
"Mastering the game of Go with deep neural networks and tree search" 


Creating a "driving" video games for deep learning A.I. to teach real world object recognition 
a) Adrien Gaidon(Research Scientist at Xerox): "You don’t just generate pixels, you also generate the supervision [AI] 


reguires. ... What I'm showing is that the technology is mature enough now to be able to use data from computers to 
train other computer programs" 


MineCraft: Potential to be used as real-world training for a.i. (easier as it has a very finite & defined potential of inputs, opposed 
to real life’s near infinite potential of inputs.) 


Super Mario (https://www.youtube.com/watch?v=qv6U VOQ0F44 Generation; Species; Genome: based off parent’s fitness score 
success. Random mutations affect movement through the level & Fitness score increases as mario progresses in level. ) 
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Evolutionary Algorithm with Example: Super Mario (https://www.youtube.com/watch? 
v=qv6UVOQoF 44&feature=youtu.be&t=1m4s) 


Source Code: http://pastebin.com/ZZmSNaHX 


CS.UTexas paper the code is based off: Evolving Neural Networks through Augmenting Topologies (http:// 
nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf) 


Population 8. Mutation chance: modifying how you want further generations 
Generic Functions: New + Get : Node, Neuron, Genome, etc. 


Interesting Functions: function mutate(genome) , function removeWeakSpecies(), generateNetwork(genome) 


Gaming A.l. Case Study: 


— 


ARENA 


QUAKE 


id Software 


A.I. In Previous First Person Shooter Games 


e Early FPS games had very basic A.I. mainly relying on the A" algorithm 
e In the 1994 game DOOM, enemy A.I. worked as follows: 
o When an enemy sees or hears a player, it will move in a straight line towards them 


o If the straight path is obstructed, it will move in a random direction for a fixed time 
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Source: http://mehm.net/blog/wp-content/uploads/2015/02/Doom Sprites.gif 


Goals of the A.I. In Quake III (1999) 


e Seeing the shortcomings of the A.l. 
in previous FPS games, id Software 
set out to create an extremely 
realistic form of A.l for Quake III 


e [heir major goals with this project 
were to make a bot that could: 


o Act like a human player and be hard to 
distinguish from one 


Source: http://www.ericspitler.com/images/3d/visor_final01.jpg 


o Navigate its environment easily and pick 
up items and use weapons 


Ouake's A.I. Network 


e [his network shows the 
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Does not divert bot very much. 


Design of A.I. In Quake III (1999) 


e The high-level design of Quake's A.I. is made up of 4 layers that interact with 
each other. 
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Source http://fd.fabiensanglard.net/quake3/The-Quake-Ill-Arena-Bot.pdf 


Design of A.I. In Quake III (1999) 


e LEVEL 1 a" 


Team leader Al 
na 


This is the input/output level r 


The bot sends information 

about the environment to 

higher layers and receives Gi 
movement information from 

higher layers 


Design of A.I. In Quake III (1999) 
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This is the intelligence layer 3“ 


Uses fuzzy logic to select 

goals, navigates its 

environment, and chats with ‘a 
other players 


Fuzzy Logic In Quake III 


e To act human, there needs to be a way for the 
bot to set goals and decide how to act on them 


e For this, id used fuzzy logic as a way for the 
bot to express how much it wants certain 
weapons or how important certain actions were 


to it. 


o The bot can assign a high fuzzy value to retrieving more 
ammo for a powerful weapon it already has which is low 
on ammo 


e A genetic algorithm is used to make sure the 
Source: http://i301.photobucket.com/albums/nn51/rager825/ 
| | in QuakeArmory.png 


Design of A.I. In Quake III (1999) 


e LEVEL 3 
This is the production rules layer 


Has various states describing 
the situation the bot is in and is 
able to reason about what 
actions the bot should take. 
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Production Rules in Quake III (1999) 


e Uses a rule-based system with rules in the form IF (condition) THEN 
(action) just like in procedural programming 


o The condition is a logical expression of facts from the knowledge base 


o The action operates on these facts 


e Quake extracted and stored the expertise from human players so that the 
actions could accurately portray what a person would do in a situation 


An example production rule: 


e IF bot is fighting AND low on health AND does not have good weapon THEN 


retreat from the fight 
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Design of A.I. In Quake III (1999) 


e LEVEL 4 
This is the team leader layer 


Used for game types where the 
bot will be a team leader and 
need to coordinate a strategy to 
accomplish tasks 
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Putting it All Together 


e All of this allows the bots in Quake to be very lifelike 


o All 3 of the original design goals were met 


e Bots can interact with all parts of the environment and even solve simple 
puz 


Source: http://fd.fabiensanglard.net/quake3/The-Quake-Ill-Arena-Bot.pdf 


What is Starcraft? | 


- Real Time Strategy (RTS), 
similar to Warcraft Ill, developed 
by Blizzard 


- Was the most popular game in 
E-Sports before League of 
Legends 


- Win Condition: Destroy all 
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Ihree Races i 


http://i.stack.imgur.com/LA6AZ. gif 


Three Races 


http://i.stack.imgur.com/66FDZ jpg 


Larger Army / Unit Composition 


A 447, 


http://www.sc2blog.com/wp-content/uploads/2008/06/terran-versus-zerg.jpg 


Unit Counters 
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Simplistic Starcraft Models 
Army control and production 
- Bigger / Stronger army wins 
- Unit Counters (Air vs Ground) 


Economy Maintenance 


- Resources require to produce units 


More resources faster unit production, which means 
laraer army 


Advantages 


Computer Advantages: gifs.com/gif/korean-gamers-apm- 
demonstration-SWObO9 
- Actions Per Minute (APM) 
- 300-500 vs 1000s 
- Perfect Execution 


- Multi-tasking 


- Micro and Macro 


Designing a Solution 


1. Immense amount of states: 
a. Chess: 10450 states 


b. Go: 10"170 states 


2. Partially Observable 


a. Fog of War 


3. Real Time 


a. Constant changes in states 


UAlbertaBot - Open Source 


Al / Bot, written in C++ to play Starcraft Brood War 
Uses the BWAPI: The BroodWar API 


Written by David Churchill, Assistant Professor of 
Computer Science, at Memorial University of 
Newfoundland 


= = 
Design and Architecture 
Module 
Combat Worker Information 
Commander Manager Manager 


Production 
Manager 


Strategy z 
Manager MapTools MapGrid 
Building DistanceMap 


Manager 


BuildOrder 
Search 


BuildOrder 
Queue 


Micro Combat BOSS Building 
Manager Simulation Project Placer 
Melee Ranged Detector MicroSearch 
Manager Manager Manager Manager 


Logic Flow 


CombatCommander.update() 
- Scout to find enemy unit, if ready 
- If enemy in our region send defence squad, produce units if 
needed 


- If offensive units are available and sufficient in size 
- Attack known enemy base 
- Attack visible units 
- Attack closest enemy buildings 
- Explore until new target is found 


Logic Flow 


Squads.update() 


- Perform CombatSimulation 

- If simulation returns victory then engage 

- Call MicroManager specific to each type of unit 
- Else 


- Returns loss then retreat to base 


The Reality 


While (win !- true) 4 
Action = ask("How can I create win condition?”); 
execute(action); 
} 
/* Multiple Win Conditions 
- Economic Advantage 
- Have Stronger Army 


~= z Outplav / Outmaneuver */ >>>) 


Why not Deep Learning? 


A general model to win exists for games such as Go, Chess, etc. 
- Make moves to maximize the probability of winning 

A general model does not exists for RTS 
- Multiple models can work 


- Hidden Information, fake Information (Deceptions), processing new 
information 


- Humans are able to adapt much guicker and exploit weaknesses 


DeepMind and Starcraft II 


Blizzard opens Starcraft Il to Al and Machine Learning Researchers 
- Many subproblems are related to real life problems 


- Management of resources for production is very similar to industrial scale 
production 


- Decision making and adaptation under constant state changes with 
uncertainty closely resembles real world scenarios 


https://deepmind.com/blog/deepmind-and-blizzard-release-starcraft-ii-ai-research- 
environment/ 


Demo of API Developed With Blizzard 


https://www.youtube.com/watch?v=5iZlrBqDYPM 


Conclusion 


The possibility of Al taking over is still years away 


- Humans are capable of understanding a system and 
exploiting its weakness way guicker than Al 


- Human intelligence is not completely reproducible 
- Alis extremely power at performing certain tasks 


- Alis evolving rapidly 
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